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A concentration graph associated with a random vector is an undirected graph where each 
vertex corresponds to one random variable in the vector. The absence of an edge between any 
pair of vertices (or variables) is equivalent to full conditional independence between these two 
variables given all the other variables. In the multivariate Gaussian case, the absence of an edge 
corresponds to a zero coefficient in the precision matrix, which is the inverse of the covariance 
matrix. 

It is well known that this concentration graph represents some of the conditional indepen- 
dencies in the distribution of the associated random vector. These conditional independencies 
correspond to the "separations" or absence of edges in that graph. In this paper we assume that 
there are no other independencies present in the probability distribution than those represented 
by the graph. This property is called the perfect Markovianity of the probability distribution 
with respect to the associated concentration graph. We prove in this paper that this particular 
concentration graph, the one associated with a perfect Markov distribution, can be determined 
by only conditioning on a limited number of variables. We demonstrate that this number is 
equal to the maximum size of the minimal separators in the concentration graph. 

Keywords: conditional independence; graphical models; Markov properties; separability in 
graphs; undirected graphs 

1. Introduction 

Let {X, G, J-) be a triple where = X aeV'^a is a product probability space, G = (V, E) 
is a graph with a finite set of vertices V and a set of edges E CV xV in which a certain 
separation criteria C is defined, and is a family of probability distribution of random 
vectors X = {Xa,a G V)' with values in X. The triple {X,G,T) is called a graphical 
model if it satisfies the following property called the global Markov property. 

Let A, B and S be three disjoint subsets where A and B are non-empty. If S separates 
A and B according to the criteria C in G, denoted by A^-cB\S, then the random vectors 
Xyi and Xs are independent given X5, where Xyi, X^ and X5 are subvectors of X 
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indexed respectively by the subsets of vertices A, B and S. So 

ALcB\S then Xa-LLXb|Xs. (1) 

Note that the graph G should not contain loops - that is, an edge linking one vertex 
to itself - and any pair of vertices in G is connected at maximum by one edge, that is, 
there are no multiple edges between any given pair of vertices. 

When the graph G has only undirected edges, that is, 

(a,/3)6£; ^ {fi,c)eE, 

the associated graphical model is called a concentration graph model (see Lauritzen 
(1996)). Dempster (1972) studied concentration graph models for Gaussian distribu- 
tions under the name of covariance selection or covariance selection models. The ab- 
sence of an edge between a given pair of vertices (a, /3) in the associated graph indi- 
cates that the random variable Xa is independent of given all the other variables 
X_Q^ = {X^,j 7^ a,/?)'. These models are very well studied, especially the ones cor- 
responding to Gaussian distributions (see Whittaker (1990), Lauritzen (1996), Edwards 
(2000) and, recently, Letac and Massam (2007) and Rajaratnam et al. (2008)). The sepa- 
ration criteria defined on such graphs is a simple separation criteria on undirected graphs: 
S CV separates two disjoint non-empty subsets A and S of y if any path joining a vertex 
in A and another in B intersects S. 

Other graphical models are represented by graphs with bi-directed edges. These models 
are called covariance graph models. The absence of an edge between a given pair of 
vertices (a,/?) implies that Xa is marginally independent from Xp, denoted X^lLXp. 
The separation criteria in bi-directed graphs can be defined as follows: If A, B and 
S are three disjoint subsets of V , where S could be empty, the subset S separates A 
and B in the bi-directed graph G \i V \ {AVJ B S) separates A and i3, that is, any 
path connecting A and B intersects V \{Ayj BVJ S). In this paper this graph will be 
represented by non-directed edges and will be denoted by Go- So the global Markov 
property on Go, also called the covariance global Markov property, can be defined as 
follows (see Chaudhuri et al. (2007)): 

\iV\{A\JB\JS) separates A and B in Go, then X^ -LLXijIXs. (2) 

Let P be a probability distribution belonging to a certain graphical model {X,G,T). 
The probability distribution P is said to be perfectly Markov to G if the converse of the 
global Markov property (1) is also satisfied, that is, for any triple of disjoint non-empty 
subsets {A,B,S) where S is not empty, 

AUB\S ^ X^^XbIXs. (3) 

It was conjectured in Geiger and Pearl (1993) that for any undirected graph G we can 
find a Gaussian probability distribution P that is perfectly Markov to G. In the Gaussian 
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case the perfect Markovianity assumption is equivalent to the fohowing property: For all 
non-adjacent vertices a and (3 in V and for all 5* C 1/ \ {a, (3} and S ^ 0, 

5 separates a and /3 Xq,_LLX^|X5. (4) 

In this paper we will consider an undirected graph G = {V, E) and a probability distri- 
bution P that is perfectly Markov to G. Hence, if X = {Xa,a G V)' is a random vector 
with distribution P, then G satisfies the following condition 

and if A, B and S are three disjoint non-empty subsets of V 

S separates A and _B in G X^i llXslXs. 

The aim of this paper is to prove the existence of a relationship between the cardinality 
of the separators in G and the maximum number of conditioning variables needed to 
determine the full conditional independencies in P. We proceed first by defining a new 
parameter for undirected graphs, called the ^^separability order" . Subsequently we prove 
that when we condition on a fixed number of variables equal to this separability order, 
the graph that is obtained is exactly the concentration graph. 

The paper is organized as follows. Section 2 is devoted to the definition of this pa- 
rameter and of some properties thereof. In Section 3, we will define a sequence of undi- 
rected graphs constructed due to conditional independencies for a given fixed number 
k £ {0, . . . ,\V\ — 2}. More precisely, we define the fc-graph Gk = {V, Ek) by 

{a,P)^Ek ^ 3SCV\{a,l3} such that |5| = fc and X„ _LLX^|Xs. (5) 

When fc = 0, the conditional independence given an empty set corresponds to the 
marginal independence between Xa and Xp. In the case when fc = 0, the correspond- 
ing fc-graph is then denoted by Go and constructed using the pairwise Markov property 
with respect to bi-directed graphs (see Cox and Wermuth (1996) and Chaudhuri et al. 
(2007)). We mean that 

a T^Go P -LL Xp. 

The graph Go is also called a covariance graph (see Chaudhuri et al. (2007)). Wille and 
Biihlman (2006) define a graph called a 0-1 graph, which corresponds to a graph with a 
set of edges equal to EoDEi. We will show later (see Lemma 7) that this graph is equal 
to Gi. Castello and Roverato (2006) consider so called fc-partial graphs G^ = {V,El), 
which are defined as follows: 

a9^Gfc/3 ^ 3SCV\{a,P} such that jS"] < fc and X„ II X^jXg. (6) 

Obviously for a fixed fc, E^ C Ek- We will show later (see Lemma 8) that the fc-partial 
graph is equal to Gk- The principle result we prove in this paper (see Theorem 4) is 
that ii'fc C ■ • • C iJx C iJg and that G is equal to Gk, where fc is the separability order of 
G. The main assumption of this result is that probability distribution P of the random 
vector X is perfectly Markov to G. 



1182 



D. Malouche 



2. Separability order 

An undirected graph G = (V, E) is a pair of sets where V is the set of vertices and E is 
the set of edges that is a subset oi V x V, where 



For a, /3 G y , we write a ~g P when a and /? are adjacent in G, that is, (a, /?) G £'. 
A complete graph is a graph where all the vertices are adjacent, and an empty graph 
is a graph where the set of edges is empty, that is, E ^ 0. A path between a pair of 
vertices (a,/3) is a sequence of distinct vertices ao,ai, . . . ,a„ such that ao = a, = /? 
and ai Ofi+i for all i = 0, . . . ,{n — 1). 

Let [/ C y . A subgraph of G induced by U is an undirected graph Gjj = {U, Ejj) such 
that Eu = U xUnE. 

A connected graph is a graph G where any pair of vertices can be joined by a path. 
So a connected component of a graph is a subset of V that induces a maximal connected 
sub-graph of G, i.e., G C y is a connected component of G if Gc is a connected subgraph 
of G and, for any a EV\C,Wf3 gC, there is no path between a and /3. 

For any non-adjacent vertices a /3 in a graph G and for any 5 C t/ \ {a, /?}, we say 
that 5 is a separator of a and /? in G if all the paths between a and /? in G intersect 
5. Consequently, any S' D S and S" C F \ {a,/3} is also a separator of a and /3. The 
separator 5 is called a minimal separator of a and /3 if, for any S' C S and S' ^ S, S' 
cannot be a separator of a and /3. We denote by msG(a, /3) the set of minimal separators 
of a and /3 in G. It is clear that the set msG(a,/3) = if and only if a,/3 are in two 
different connected components of G. 

Let us now give the definition of the separability order of an undirected graph: 

Definition 1. The separability order of a given graph G ~ {V, E) is 



if G is not complete, and so(G) — +oo if G is complete. 

Note that complete graphs have a separability order of infinity. Also, empty graphs, 
that is, graphs with no edges between the vertices of G, have a separability order equal 
to zero. Conversely, if so(G) = then either G is composed only of complete connected 
components or G is an empty graph. We also note that the separability order is purely 
a graph-theoretic concept. 

We now give an example and proceed to prove basic properties of the separability order 
(see Lemma 1). 

Example 1. The graph in Figure 1 is an undirected graph containing five vertices. Its 
separability order, so(G), is equal to 2. We can easily see that 





so(G) 



max minllS*!, S £ msG(a, /?)} 

q/g;3 



(7) 



msG(l,3) = {{2}}, 
msG(l,4) = {{2}}, 



msG(2,5) = {{3,4}}, 
msG(l,5) = {{3,4},{2}}. 
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Hence so{G) = 2. The degree of the graph G, d{G), defined in (10) is equal to 3. 
Lemma 1. Let G — {V,E) be an undirected graph with connected components 

Gi = {VuEi),...,Gs = {Vs,Es), 

with s >2, and so(G) = m. Then 

(i) m = if and only if all the connected components of G are either complete or 
singletons. 

(ii) 771 — +CXD if and only if G is complete. 

(iii) When m> 0, the following properties are satisfied: 

1. There exists a pair (a,/3) of non- adjacent vertices and a minimal separator S 
of this a and /? such that \S\ — m. 

2. For any pair of non-adjacent vertices there exists a separator of these two 
vertices with cardinality equal to m. 

(iv) When m > 0, the separability order, so{G), is equal to the maximum separability 
order among all its non-complete connected components: 

so(G') = niax{so(Gi), G; non- complete}. (8) 

Proof. The proof of items (ii) and (iii) follows immediately from Definition 1. 

(i) If m = 0, this means that for any a /g /3 the only separator of these vertices is the 
empty set. Hence a and (3 belong to different connected components. Moreover, 
in each connected component of G there are non-adjacent vertices, since m = 0. 
Hence all the connected components of G are either complete or singletons. The 
converse of this statement follows easily from the definition of the separability 
order. 

(iv) Let us define the pairwisc separability order of a given pair of vertices a and f3 

so(a,/3|G) = min{|S'|, 5* G msG(a,/3)}. (9) 
Now let Gi , . . . , G; be the sequence of non-complete connected components of 

G. 

Now so(a, /? I G) = 0, if a G Gi and /3 G Gj when i^ j and i, j G {1, . . . , /}. 



3 




1 2 5 




4 



Figure 1. 30(6") = 2 and d{G) = 3. 
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Thus we can focus on the pairwise separabihty order of pairs within non- 
complete connected components. Then 

so(G) = max minds*!, S* £ msG(a,/3)} 

= max max minllSI, S* G mS(7, (a, /3)| 

a,P£Vk.k=l,...JaTCG^f} 

= max max minHSI, S g msg^ (a, /?)} 

fc=l,...,i QT^Gfc^* 

= max sofGfe). i-i 

k=l,...d L-l 

It is important to note that the separability order defined in this paper is exactly equal 
to the outer connectivity of the missing edges defined by Castello and Roverato (2006) 
for connected graphs. We can also prove that the separability order of a non-complete 
undirected graph G is always smaller than its degree (Lemma 2 below). 

Lemma 2. Let G — {V,E) be a non-complete undirected graph; then 

so{G)<d{G), 

where 



d(G) =maxd(a|G'), (10) 



where d{a\G) ~ |{7, a 7}| • 



Proof. Let a be a vertex in V, and let V{a\G) be the set of vertices adjacent to a. So 
d{a\G) = \V{a\G)\, the degree of a in G. Let /3 be a vertex non-adjacent to a. It is easy 
to see that V{a\G) and V{f3\G) are also separators between a and /3. Also it is easy to see 
that V(a|G) always contains one minimal separator between a and (3. For example, the 
set of vertices 7 denoted by S{a\G) that are simultaneously adjacent to a and belonging 
to one path between a and /3 is a minimal separator of a and /3. If we suppress one 
vertex from this S{a\G), this latter set will no longer be a separator between a and (3. 
The same thing also occurs for V(/3|G). Hence 

min{|S'|, S* e msG(a, /?)} < max(d(a|G), d(/3|G)). 

Then, 

so(G) = max{min{|5|,5 e msG(Q!,/3)}} < max max(d(a|G), d(/3|G)) 

= max d(a I G)=d(G). 

So, so(G) <d(G). □ 



We now define the degree two of an undirected graph. 
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1 3 




2 4 

Figure 2. d2(G) = 1, d(G) = 3, so(G) = 1. 



Definition 2. Let G = (V, E) be an undirected graph. The degree two of a vertex in G 
is defined by, 

d2(a|G) = |{7,a~G7 and d{-i\G) >2)\ 
and the degree two of the graph G, d2{G), is 

d2{G) = niaxd2(Q!|G). 

We give an example to illustrate the degree two of a simple undireeted graph. 

Example 2. The graph in Figure 2 has d2{G) = 1, d{G) = 3, so(G) = 1. 

It is easily seen that in practice the computation of the separability order is an NP- 
complete problem. The degree two of a graph could be a good upper bound for this 
separability order, as this quantity is more easily computable. We prove that so(G) < 
d2{G) in Lemma 3 below. 

Lemma 3. Let G ~ {V,E) and G' = {V,E') be two undirected graphs. Then 
(i) ifE<ZE', thend2{G)<d2{G'), 

ill) if G is connected and non-complete so(G) < d2(G). 
Proof. 

(i) First let us define V2(a|G) as follows: 

V2(a|G)=={7,a-G7 and d(7|G) > 2}. 

Now for any a(zV, V2(a|G) C V2(a|G'). Hence |V2(a|G)| < |V2(a|G')| and thus 
d2{a\G) < d2{oi\G'). This inequality is valid for any a and taking the maximum 
on a on either side gives d2{G) < d2{G'). 

(ii) If so(G) = m, then, using Lemma 1. part (iii), there exist a and j3 such that a /3 
and a minimal separator S with cardinality \S\ = m. Now V2(a | G) contains the 
set S{a\G\ which is a minimal separator between a and /3 (as defined in the proof 
of Lemma 2). As 5 is the smallest minimal separator of a and /3, 

m < \S[pL\G)\ < \V2{a\G)\ < d2{G) 

since (i2(G) = maxcgy |V2(q!|G)|. Hence so(G) < (i2(G). '-' 
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3. Concentration graph by low-order conditioning 

As before, let G — (V, E) be an undirected graph with the set of vertices V and the set 
of edges E. Let X = XaeV'^a be a product probabihty space. The aim of this section 
is to prove the following result. 

Theorem 4. Let (X, G, T) he a concentration graph model and P a probability distri- 
bution belonging to T . Let us consider for any k G {0, . . . , \V\ — 2} the undirected graph 
Gk = {V,Ek) constructed as described in (5); 

a'/^GkP 3S'cy\{a,/3} such that \S\~ k and l]-Xfj\'K.s. 

Suppose that P is perfectly Markov to G and so(G) = m, then 

E = Era C C • • ■ C ij^ . 

Furthermore, if so (Gq) < \V\ — 2, then Ei C Eq. 

Theorem 4 will be proved using the following series of lemmas. 

Lemma 5. Let {X,G,J-) be a concentration graph model and let P be a probability 
distribution belonging to T . Suppose that G is connected and non-complete and so(G') = m 
where m > 0. Suppose also that P is perfectly Markov to G. Then G = Gm, where Gm is 
the undirected graph constructed using (5). 

Proof. Let a and /3 be two vertices and let us consider a random vector X = {Xa, a gV)' 
with distribution P. For any pair (a,/?) such that a ^g from Lemma l(iii), there 
exists a non-empty subset S with cardinality equal to m that is a separator of a and 
/3. Using the global Markov property with respect to G (see (1)), we can conclude that 
Xq, -I-LX/3IX5. Using (5) we conclude that a '^g^ P- Since this is valid for any pair (a, /3) 
we can conclude that E^ ^ E. 

Conversely, suppose that a t^g,^ /?; then there exists a separator S* C F \ {a, /?} with 
cardinality m such that Xa -LLX^IXg. Using the perfect Markovianity property we can 
say that S separates a and /3 in G. Thus we can assert that a ^g P- Since this argument 
is valid for any (a,/3) we can conclude that E C Em- 

We have altogether shown that Em ^ E and E C E„i , hence E = Em , and thus G = 

Gm- n 

Lemma 6. Let {X, G, J-) be a concentration graph model and let P be a probability distri- 
bution belonging to T . For m,k €zN, let Gm md Gk be two undirected graphs constructed 
using (5). // P is perfectly Markov to G and l<m<k<\V\^2 then E^ C Em- 

Proof. Let a and /3 be two vertices. Let us consider a random vector X = {Xa, a G V)' 
with distribution P- Suppose that a 9^g„ then there exists a separator S ^V\ {a, /?} 
with cardinality m such that Xa lLXp\yis- By the perfect Markovianity property we 
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can conclude that S separates a and /3 in G. But since k>m,we can find a subset S' of 
V\{a,l3} containing 5* with cardinahty k such that S' is a separator of a and /3 in G. 
Using the global Markov property, we determine that Xa IL XplX-s' ■ Hence a P- 

Since a t^g^ P imphes that a i^Gu P ^^'^ P^i^ {(^iP)j we can conclude that Q 
E^. " □ 

Lemma 7. Let P be a probability distribution in X . Let Gq = {V,Eo) and Gi = {V,Ei) 
be two undirected graphs constructed using (5) for fc = 0, 1, respectively. If Go is connected 
and so(G'o) = mo < \V\ — 2, then Ei C Eq. 

Proof. Let a and /3 be two vertices and let us consider a random vector X = {Xa, a GV)' 
with distribution P. Suppose that a P- By assumption so(G'o) = tuq, hence there 
exists a subset S' of \ {a, (3} for which |5| = toq. Let 7 e F \ (5 U {a, /?}), which is not 
empty because mo < | ^ | — 2 . Then \ {a, /3, 7} contains S and so it is a separator of a and 
/3 in Gq. So {7} m-separates {a} and {/?} in Gq. Here Go is seen as an ancestral graph. 
Using the covariance global Markov property (2) with respect to bi-directed graphs (see 
Cox and Wermuth (1996), Chaudhuri et al. (2007), or Drton and Richardson (2008)), we 
can conclude that X^ ^^-X|3\X^. Then a ^Gi P- Hence Ei C Eq. □ 

It is easily seen that the results in Lemmas 5-7 lead to the proof of Theorem 4. 
Castello and Roverato (2006) prove, by also assuming the perfect Markovianity, that the 
concentration graph model G is equal to the fc-partial graph G^ as defined in (6) when 
k is smaller than the separability order of G, referred to in that paper as the "outer 
connectivity of the missing edges". The result in Theorem 4, however, is based on a 
construction of a sequence of nested graphs. It starts from the covariance graph, that is, 
Go, the 0-graph, and it becomes stationary and equal to the concentration graph when 
the number of conditioning variables is equal to the separability order of the concentration 
graph. 

In Lemma 8, we show that /c-partial graphs and /c-graphs are equal when the perfect 
Markovianity assumption is satisfied. Next, we give a corollary of Theorem 4. In Corol- 
lary 9 we give a condition that allows us to determine the last undirected /c-graph in 
the sequence of nested graphs obtained due to Theorem 4. This condition is given in 
term of the degree two of the fc-graphs, not as in Castello and Roverato (2006), where 
this condition is expressed as a function of the outer connectivity of connected edges, a 
quantity that can be difficult to compute. 

Lemma 8. Let X = {Xa,a G V)' be a random vector with distribution P belonging to 
a graphical model generated by an undirected graph G~ {V,E). The undirected graphs 
G^ = (y, E^) and Gk = (V, Ek) are respectively the k-partial graph and the k-graph defined 
as in (6) and (5). // P is perfectly Markov w.r.t. to G then for any k G {1, . . . ,\V\ — 2}, 
we have E C_ E^ ^ E^ . 

Proof. By definition it is easily seen that a i^G^ P implies a ^qp /?, hence Ek C E^. 
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Now let us assume that a i^G\ then there exists S' C V \ {a,/?}, 15*1 < fc such that 
Xq, 11X^1X5 = (-^7,7 G S")'. If l^l = fc, the problem is solved. If jS"] < fc, using the perfect 
Markovianity of P we can say that S separates a and /3 in G. Then we can construct 
an iS" C y \ {a,/3} with |S"| = fc, S' ^ S such that S' separates a and (3 in G. We can 
now use the global Markov property (see (1)), to assert that Xa ILXplX-s' and hence 
ct i^Gk P- We can therefore deduce that C Ek- Since Ek Q E^ and E^ C Ek we can 
conclude that E^ = Ek - The inclusion E C E'j. has already been proved in Theorem 4. □ 

We can also deduce the following corollary from Theorem 4. 

Corollary 9. Let (X,G,J-') be a concentration graphical model such that G is a non- 
complete connected graph and let P be a probability distribution belonging to T . Let us 
consider for any k € {0, . . . , |F| — 2} the undirected graph Gk = {V, Ek) constructed as 
described in (5). Let us assume that P is perfectly Markov to the graphical model G and 
d2{G) < \V\-2. Then there exists k e {1, . . . ,\V\ - 2} such that 

d2{Gk)<k and G = Gk. (11) 

Proof. Let us assume that for all k G {1, . . . ,\V\ — 2} that d2{Gk) > k. This implies 
for example that d2(G'iy|_2) > \V\ — 2. As the concentration graph G is exactly the 
G\v\-2 we deduce that d2{G) > \V\ — 2 which is a contradiction with our assumption, 
i.e., d2{G) < \V\ — 2. Hence there exists an integer k such that d2{Gk) < k. But E C Ek 
and, applying Lemma 3(ii), we deduce that so(G') —m< d2{G) < d2{Gk) < k. 

Using Theorem 4, as fc > to, we can conclude that G = Gm = Gk- □ 

Corollary 9 can be useful if we wish to determine the concentration graph from a given 
data set when assuming perfect Markovianity. It is sufficient to check the degree two of 
each estimated fc-graph. 

4. Conclusion 

In this paper we have proved that a concentration graph model can be determined using 
a limited number of conditioning variables. The cardinality of this limited subset is 
determined by looking at the structure of the undirected graph associated with the 
corresponding distribution global Markov property. Certainly the perfect Markovianity 
assumption is also needed for our result to be valid. Our result remains true for both 
continuous and discrete distributions. 

Our result can also be used as a justification of the estimation of graphical mod- 
els by low-order conditioning such as using the PC algorithm (see Spirtes et al. 
(2000), Kalisch and Biihlmann (2007), Kjaerulff and Madsen (2007)), the 0-1 procedure 
(see Friedman et al. (2000) and Wille and Biihlman (2006)), or the gp-procedure (see 
Castello and Roverato (2006)). Practical applications of these procedures are useful when 
the number of observations are far fewer than the number of variables. We first estimate 
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the sequence of nested graphs Gk starting from fc = (test on marginal independence 
between variables) and proceed accordingly. This procedure is terminated when the num- 
ber of conditioning variables becomes greater than the degree two of the estimated graph 
Gk- In this sense the theory above has tremendous scope for applications. 
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